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IMAGE PROCESSING APPARATUS AND METHOD, PROGRAM, AND 
RECORDING MEDIUM 

BACKGROUND OF THE INVENTION 

The present invention relates to an apparatus, a method, a program, and a 
recording medium for image processing and, particularly, to an apparatus, a method, a 
program, and a recording medium for image processing that, in converting an 
interlaced scanning image into a progressive scanning image, improve vertical 
resolution in a still image area, smoothly interpolate an edge in a moving image area, 
and prevent an error in the conversion. 

Techniques for converting an interlaced scanning image into a progressive 
scanning image (IP conversion) are widely spreading. 

The interlaced scanning system scans every other scanning line of an actual 
display image (performs scanning by skipping every other scanning line). One image 
scanned by the interlaced scanning system is referred to as a field. When continuous 
images are displayed, adjacent fields have pixels displaced from each other by one 
scanning line, and such continuous fields are displayed alternately. 

As opposed to the interlaced scanning system, the progressive scanning 
system scans all scanning lines of the display image.Hence, the process of converting 
an interlaced scanning image into a progressive scanning image is that of interpolating 
pixels on lines with no pixels in the interlaced scanning image using originally present 
pixels, and thereby generating pixels on all the scanning lines. 

Conventional interpolation methods include a method referred to as double 
writing, which scans each scanning line twice by display with a delay corresponding 
to one scanning line, and a method of linear interpolation using pixels in a vertical 
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direction. 

In addition, a method has been proposed which uses information of a plurality 
of fields to enhance vertical resolution. Since, in the case of continuous fields of the 
interlaced scanning system, a field immediately preceding a current field is displaced 
by one scanning line in relation to the current field, as described above, a pixel on the 
immediately preceding field is present at a position to be interpolated on the current 
field. Specifically, when the pixel to be interpolated is a still image pixel, the pixel 
on the immediately preceding field is used as it is for interpolation, and in the case 
that the pixel to be interpolated is a moving image pixel, the pixel to be interpolated is 
interpolated by linear interpolation using an upper and a lower pixel on the current 
field. 

However, when the above double writing method is used, for example, an 
image representing an oblique edge is displayed as large steps as a result of the IP 
conversion. 

In addition, although the linear interpolation improves the display image as 
compared with the double writing, the linear interpolation may result in a slightly 
blurred display. Furthermore, as in the case of the double writing, an oblique edge is 
displayed as steps as a result of the linear interpolation. 

In order to prevent a stepwise display of an edge in an oblique direction, a 
method has been proposed which carries out IP conversion while detecting the 
direction of the edge. The method is disclosed in Japanese Patent Laid-Open No. Hei 
3-42832. The method determines the differences in absolute values of three sets of 
pixels; i.e., a set of an upper and a lower pixel, a set of an upper right and a lower left 
pixel, and a set of a lower right and an upper left pixel with a remarked pixel to be 
interpolated as a center, and then interpolates the remarked pixel by using a 
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combination of pixels having the lowest value. When all the values are higher than a 
predetermined value, the method interpolates the remarked pixel by copying the value 
of the upper pixel. Thus, even when an oblique edge is present, the method enables 
smooth interpolation in a direction of the oblique edge. 

However, although the method of carrying out conversion while detecting the 
direction of the edge improves the display of the oblique edge, the method 
unfavorably lowers vertical resolution in a still image area as compared with the 
method using a plurality of fields. Moreover, when applied to an actual image, the 
method of carrying out conversion while detecting the direction of the edge may 
calculate correlation in the oblique direction somewhat higher than in the vertical 
direction, depending on noise or conditions of illumination, for example. This tends 
to result in an error in which an actual pattern continuous in the vertical direction is 
not displayed correctly. 

In addition, a common video signal or the like includes only a slight high- 
frequency component, and thus an edge appears only as a blurred image. Therefore, 
it is difficult to detect the direction of a local edge accurately, and even when an edge 
in a specific direction is to be enhanced, a plurality of edge directions are detected as 
candidates, which renders selection impossible. As a result, the image can be 
generated only with the same level of image quality as that of the conventional linear 
interpolation. 

SUMMARY OF THE INVENTION 

The present invention has been made in view of the above, and it is 
accordingly an object of the present invention to improve vertical resolution in a still 
image portion, smoothly interpolate an edge in a moving image area, and prevent a 
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conversion error in the conversion of an interlaced scanning image into a progressive 
scanning image. 

According to the present invention, there is provided an image processing 
apparatus including: an inter-field interpolating means for interpolating a pixel of an 
image of a progressive system using a pixel of a field of an image of an interlacing 
system to be converted and a pixel of a different field; an intra-field interpolating 
means for interpolating a pixel of the image of the progressive system using pixels 
within the field of the image of the interlacing system to be converted; a determining 
means for determining whether a pixel to be interpolated of the image of the 
progressive system is a moving image pixel or a still image pixel; and a selecting 
means for selecting either the pixel interpolated by the intra-field interpolating means 
or the pixel interpolated by the inter-field interpolating means on the basis of a result 
of the determination by the determining means. 

The image processing apparatus can further include an edge enhancing means 
for enhancing an edge of the image of the interlacing system to be converted. 

The edge enhancing means can subject each of pixels of the image of the 
interlacing system to horizontal-direction one-dimensional filter processing and 
vertical-direction one-dimensional filter processing to enhance thereby the edge of the 
image. 

The image processing apparatus can further include a central pixel energy 
calculating means for calculating vertical-direction central pixel energy and 
horizontal-direction central pixel energy around each of the pixels from pixel values 
around the pixel, and the edge enhancing means can perform the filter processing on a 
pixel whose vertical-direction central pixel energy and horizontal-direction central 
pixel energy calculated by the central pixel energy calculating means exceed a 
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predetermined value. 

The inter-field interpolating means can interpolate the pixel by copying the 
pixel of the different field onto the field of the image of the interlacing system to be 
converted as it is. 

The image processing apparatus can further include a correlation detecting 
means for detecting the correlation between pixels at positions symmetrical with 
respect to a point of the pixel to be interpolated within the field of the image of the 
interlacing system to be converted, the pixels being on horizontal lines over and under 
the pixel to be interpolated, and the intra-field interpolating means can interpolate the 
pixel using a combination of pixels having the strongest correlation with each other 
detected by the correlation detecting means. 

The correlation detecting means can detect, as correlation, an absolute value 
of a difference between pixel values of the pixels at positions symmetrical with 
respect to the point of the pixel to be interpolated. 

The image processing apparatus can further include a proximity energy 
calculating means for calculating the proximity energy of the pixel to be interpolated 
from pixel values near the pixel to be interpolated, and when a value of the proximity 
energy of the pixel to be interpolated calculated by the proximity energy calculating 
means is higher than a predetermined threshold value, the intra-field interpolating 
means can interpolate the pixel on the basis of the pixels having the strongest 
correlation with each other detected by the correlation detecting means. 

The proximity energy may be a sum of the absolute values of the differences 
between pixel values of a predetermined number of pixels that are present on lines 
extending in a horizontal direction over and under the pixel to be interpolated and that 
are opposite to each other in a direction perpendicular to the lines or a sum of the 
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absolute values of the differences between pixel values of a predetermined number of 
pixels that are present on the lines extending in the horizontal direction over and under 
the pixel to be interpolated and that are present at positions symmetrical with respect 
to a point of the pixel to be interpolated. 

According to the present invention, there is provided an image processing 
method including: an inter-field interpolating step for interpolating a pixel of an image 
of a progressive system using a pixel of a field of an image of an interlacing system to 
be converted and a pixel of a different field; an intra-field interpolating step for 
interpolating a pixel of the image of the progressive system using pixels within the 
field of the image of the interlacing system to be converted; a determining step for 
determining whether a pixel to be interpolated of the image of the progressive system 
is a moving image pixel or a still image pixel; and a selecting step for selecting either 
the pixel interpolated by processing of the intra-field interpolating step or the pixel 
interpolated by processing of the inter-field interpolating step on the basis of a result 
of the determination by processing of the determining step. 

According to the present invention, there is provided a program including: an 
inter-field interpolation control step for controlling interpolation of a pixel of an image 
of a progressive system using a pixel of a field of an image of an interlacing system to 
be converted and a pixel of a different field; an intra-field interpolation control step 
for controlling interpolation of a pixel of the image of the progressive system using 
pixels within the field of the image of the interlacing system to be converted; a 
determination control step for controlling determination of whether a pixel to be 
interpolated of the image of the progressive system is a moving image pixel or a still 
image pixel; and a selection control step for controlling selection of either the pixel 
interpolated by processing of the intra-field interpolation control step or the pixel 
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interpolated by processing of the inter-field interpolation control step on the basis of a 
result of the determination by processing of the determination control step. 

According to the present invention, there is provided a program on a recording 
medium, including: an inter-field interpolation control step for controlling 
interpolation of a pixel of an image of a progressive system using a pixel of a field of 
an image of an interlacing system to be converted and a pixel of a different field; an 
intra-field interpolation control step for controlling interpolation of a pixel of the 
image of the progressive system using pixels within the field of the image of the 
interlacing system to be converted; a determination control step for controlling 
determination of whether a pixel to be interpolated of the image of the progressive 
system is a moving image pixel or a still image pixel; and a selection control step for 
controlling selection of either the pixel interpolated by processing of the intra-field 
interpolation control step or the pixel interpolated by processing of the inter-field 
interpolation control step on the basis of a result of the determination by processing of 
the determination control step. 

The apparatus, the method, and the program for image processing according 
to the present invention interpolate a pixel of an image of a progressive system using a 
pixel of a field of an image of an interlacing system to be converted and a pixel of a 
different field, interpolate a pixel of the image of the progressive system using pixels 
within the field of the image of the interlacing system to be converted, determine 
whether a pixel to be interpolated of the image of the progressive system is a moving 
image pixel or a still image pixel, and select either the intra-field interpolated pixel or 
the inter-field interpolated pixel on the basis of a result of the determination. 

BRIEF DESCRIPTION OF THE DRAWINGS 
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Fig. 1 is a block diagram of an image processing apparatus to which the 
present invention is applied; 

Fig. 2 is a flowchart of assistance in explaining IP conversion processing of 
the image processing apparatus of Fig. 1; 

Fig. 3 is a flowchart of assistance in explaining one-dimensional vertical edge 
enhancement processing; 

Fig. 4 is a diagram of assistance in explaining a one-dimensional vertical 

filter; 

Fig. 5 is a diagram of assistance in explaining a one-dimensional vertical 

filter; 

Fig. 6 is a flowchart of assistance in explaining one-dimensional horizontal 
edge enhancement processing; 

Fig. 7 is a diagram of assistance in explaining a one-dimensional horizontal 

filter; 

Fig. 8 is a diagram of assistance in explaining a one-dimensional horizontal 

filter; 

Fig. 9 is a flowchart of assistance in explaining inter-field interpolation 
processing; 

Fig. 10 is a diagram of assistance in explaining the inter- field interpolation 
processing; 

Fig. 1 1 is a flowchart of assistance in explaining intra-field interpolation 
processing; 

Fig. 12 is a diagram of assistance in explaining the intra-field interpolation 
processing; 

Fig. 13 is a flowchart of assistance in explaining still/moving image area 
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determination processing; 

Fig. 14 is a diagram of assistance in explaining the still/moving image area 
determination processing; 

Fig. 15 is a flowchart of assistance in explaining selection processing; 

Fig. 16 is a diagram of assistance in explaining a one-dimensional vertical 

filter; 

Fig. 17 is a diagram of assistance in explaining a one-dimensional horizontal 
filter; and 

Fig. 1 8 is a diagram of assistance in explaining media. 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 

Fig. 1 shows a configuration of an embodiment of an image processing 
apparatus according to the present invention. Field memories 1 la to 1 If temporarily 
store information for one field of an image of an interlaced scanning system 
(hereinafter referred to as an interlaced image) inputted thereto and then output the 
information to units in a succeeding stage. Specifically, the field memory 11a stores 
image information for one field inputted at a predetermined time t, and at a next time t 
+ 1 , the field memory 11a outputs the field data stored from the time t to an image 
refresher 12a, a still/moving image area determining unit 17 and the field memory lib. 
At the same time, the field memory 11a stores new field data of the time t + 1 . The 
field memory 1 lb stores the field data of the time t inputted from the field memory 
11a at the timet + 1. 

At a time t + 2, the field memory 11a outputs the field data of the time t + 1 to 
the image refresher 12a, the still/moving image area determining unit 17, and the field 
memory lib, and stores new field data of the time t + 2. At the same time t + 2, the 
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field memory 1 lb outputs the field data of the time t to a field memory 1 lc, an image 
refresher 12b, and the still/moving image area determining unit 17, and stores the field 
data of the time t + 1 inputted from the field memory 11a. The field memory 11c 
stores the field data of the time t inputted from the field memory lib. 

Further, at a time t + 3, the field memory 11a outputs the field data of the time 
t + 2 that has been stored until the time t + 3 to the field memory lib, the image 
refresher 12a, and the still/moving image area determining unit 17, and stores field 
data of the time t + 3. At the same time, the field memory 1 lb outputs the field data 
of the time t + 1 to the field memory 11c, the image refresher 12b, and the still/moving 
image area determining unit 1 7, and stores the field data of the time t + 2 inputted 
from the field memory 1 la. The field memory 11c outputs the field data of the time t 
stored therein to a field memory lid and the still/moving image area determining unit 
17. 

Further, at a time t + 4, the field memory 11a outputs the field data of the time 
t + 3 that has been stored therein until the time t + 4 to the field memory 1 lb, the 
image refresher 12a, and the still/moving image area determining unit 17, and stores 
field data of the time t + 4 inputted to the field memory 11a. The field memory 1 lb 
outputs the field data of the time t + 2 stored therein to the field memory 11c, the 
image refresher 12b, and the still/moving image area determining unit 17, and stores 
the field data of the time t + 3 inputted from the field memory 1 la. The field memory 
11c outputs the field data of the time t + 1 stored therein to the field memory lid and 
the still/moving image area determining unit 17, and stores the field data of the time t 
+ 2 inputted from the field memory 1 lb. The field memory lid outputs the field data 
of the time t stored therein to the still/moving image area determining unit 17, and 
stores the field data of the time t + 1 inputted from the field memory 1 lc. 
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Thus, the field memories 1 la to 1 Id store four continuous pieces of field data 
at all times while transferring the field data sequentially. Field memories lie and 1 If 
store pieces of field data supplied from the image refreshers 12a and 12b, respectively, 
in which pieces of field data, an edge in a vertical direction and an edge in a horizontal 
direction are each enhanced, and then sequentially output the pieces of field data to an 
intra-field interpolating unit 14 and an inter-field interpolating unit 15. 

The image refresher 12a performs processing of a one-dimensional vertical 
filter and a one-dimensional horizontal filter on the field data inputted from the field 
memory 11a while using a buffer 13a as required, thereby enhancing edge portions in 
the vertical direction and the horizontal direction, and then outputs resulting field data 
to the field memory lie. Processing of the image refresher 12b and a buffer 13b is 
the same as above; however, the image refresher 12b and the buffer 13b perform field 
data processing with a temporal delay corresponding to one field. 

The intra-field interpolating unit 14 generates and interpolates a pixel in a 
moving image area of a progressive image from the edge-enhanced field data stored in 
the field memory lie and then outputs the pixel as a progressive image to an 
interpolation memory 16a to store the pixel therein. Details of intra-field 
interpolation processing will be described later. 

The inter-field interpolating unit 1 5 generates and interpolates a pixel in a still 
image area of a progressive image from the two continuous pieces of field data stored 
in the field memories lie and 1 If and then outputs the generated progressive image to 
an interpolation memory 16b to store the progressive image therein. Details of inter- 
field interpolation processing will be described later. 

The still/moving image area determining unit 1 7 determines from the four 
continuous pieces of field data inputted from the field memories 1 la to lid whether a 
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pixel to be interpolated of the newest piece of field data is a pixel in a still image area 
or a pixel in a moving image area and then stores the result. The processing of 
determining whether the pixel to be interpolated is a pixel in a still image area or a 
pixel in a moving image area will be described later in detail. 

On the basis of the information stored in the still/moving image area 
determining unit 17, a selector 18 reads the pixel interpolated by the intra-field 
interpolating unit 14 from the interpolation memory 16a when the pixel to be 
interpolated is a pixel in a moving image area and reads the pixel interpolated by the 
inter-field interpolating unit 15 from the interpolation memory 16b when the pixel to 
be interpolated is a pixel in a still image area. The selector 1 8 thereby generates and 
outputs a final progressive image. 

The IP conversion processing of the image processing apparatus to which the 
present invention is applied will next be described with reference to the flowchart of 
Fig. 2. 

At a step SI, the field memory 11a outputs field data stored therein to the field 
memory lib, the image refresher 12a, and the still/moving image area determining 
unit 17 and stores field data inputted to the field memory 11a. At the same time, the 
field memory 1 lb outputs field data hitherto stored therein to the field memory 11c, 
the image refresher 12b, and the still/moving image area determining unit 17. The 
field memory 11c outputs field data hitherto stored therein to the field memory lid 
and the still/moving image area determining unit 17 and stores the field data inputted 
from the field memory 1 lb. The field memory lid outputs field data hitherto stored 
therein to the still/moving image area determining unit 1 7 and stores the field data 
inputted from the field memory 11c. 

At a step S2, the image refreshers 12a and 12b perform one-dimensional 
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vertical edge enhancement processing. 

The one-dimensional vertical edge enhancement processing of the image 
refreshers 12a and 12b will be described in the following with reference to the 
flowchart of Fig. 3. Since the image refreshers 12a and 12b perform the same 
processing on two continuous pieces of field data, the processing of the image 
refresher 12a will be described in the following. 

At a step S21, the image refresher 12a determines whether the field data 
inputted from the field memory 11a includes an unprocessed pixel. When the image 
refresher 12a determines that the field data inputted from the field memory 11a 
includes an unprocessed pixel, the processing proceeds to a step S22. 

At the step S22, the image refresher 12a retrieves the unprocessed pixel and 
then calculates the vertical-direction central pixel energy of the retrieved unprocessed 
pixel. For example, supposing that there is field data as shown in Fig. 4, in which 
pixels a to e, pixels f to j, and pixels k to o are disposed on horizontal-direction lines y 
+ 1 , y, and y - 1 , respectively, vertical-direction central pixel energy Ev-h of an A area 
around the pixel h (area enclosed by a solid line in the figure) is obtained by the 
following equation: 

E v -h = |(b + c + d) - (1 + m + n)| ...(1) 

Here, b, c, d, 1, m, and n are pixel values of the pixels b, c, d, 1, m, and n, 
respectively. Thus, vertical-direction central pixel energy E v of the equation (1) is 
the absolute value of the difference between the sums of the pixel values on the upper 
line and the lower line with the unprocessed pixel at the center. Therefore, when the 
upper pixels are correlated with the lower pixels, there is no large difference between 
the pixel values of the upper pixels and the lower pixels, and accordingly the vertical- 
direction central pixel energy is low. On the other hand, when the upper pixels are 
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not correlated with the lower pixels, there is often a large difference between the pixel 
values of the upper pixels and the lower pixels, and consequently the vertical-direction 
central pixel energy is high. 

The image refresher 12a obtains the vertical-direction central pixel energy Ev-h 
of the unprocessed pixel by calculating the above equation (1). 

At a step S23, the image refresher 12a determines whether the obtained 
vertical-direction central pixel energy E v is at a predetermined value or more. When 
the image refresher 12a determines that the obtained vertical-direction central pixel 
energy Ev is at the predetermined value or more, that is, when the image refresher 12a 
determines that the unprocessed pixel has no strong correlation with the pixels in the 
vertical direction, the processing proceeds to a step S24. 

At the step S24, the image refresher 12a obtains a maximum value and a 
minimum value by comparison of the pixel values of the three vertical pixels 
including the unprocessed pixel. Specifically, when the unprocessed pixel is the pixel 
h, as shown in Fig. 4, for example, the image refresher 12a reads the pixel values of 
the vertical pixels c, h, and m (B area enclosed by a broken line in Fig. 4), including 
the pixel h, and then obtains a maximum value (c, h, m) and a minimum value (c, h, 
m) of the pixel values as shown in Fig. 5. 

At a step S25, the image refresher 12a performs one-dimensional vertical filter 
processing, as shown in Fig. 5, on the pixels c, h, and m in the B area shown in Fig. 4. 
Specifically, the processing of the one-dimensional vertical filter is (1/2 - a/2, a, 1/2 - 
a/2) (1 < a □ 2), and a pixel value hv-fiiter resulting from the filter processing is 
obtained by a calculation as expressed by the following equation (2): 

hv-fmer = c x (1/2 - o/2) + h x a + m x (1/2 - 
a/2) ... (2) 
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where a is a constant that can be set arbitrarily in a range of 1 < a □ 2, and 
thus enables adjustment of a degree of edge enhancement. 

At a step S26, the image refresher 12a compares the pixel value hy-niter 
resulting from the filter processing with the maximum value (c, h, m), as shown in Fig. 
5, and thereby determines whether the pixel value h V -futer resulting from the filter 
processing is more than the maximum value (c, h, m). When the image refresher 12a 
determines that the pixel value hv-mter resulting from the filter processing is more than 
the maximum value (c, h, m), the image refresher 12a replaces the pixel value hv-niter 
with the maximum value (c, h, m) at a step S27. 

At a step S28, the image refresher 12a stores in the buffer 13a the maximum 
value (c, h, m) replacing the pixel value h V -mter as the pixel value of the pixel h. The 
processing then returns to the step S21 to repeat the same processes until it is 
determined that the one-dimensional vertical edge enhancement processing is 
performed on all the pixels. 

When the image refresher 12a determines that the vertical-direction central 
pixel energy Ey is not at the predetermined value or more, that is, when the image 
refresher 12a determines that the unprocessed pixel has a strong correlation with the 
pixels in the vertical direction at the step S23, the processing proceeds to the step S28. 
The image refresher 12a then stores the pixel value of the pixel h in the buffer 13a, as 
it is without having filter processing performed thereon. The processing returns to 
the step S21 to repeat the processes from the step S21 on down. 

When the image refresher 12a determines at the step S26 that the pixel value 
hv-fiiter resulting from the filter processing is not more than the maximum value (c, h, 
m), the image refresher 12a compares the pixel value hy-fiiter resulting from the filter 
processing with the minimum value (c, h, m), and thereby determines whether the 
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pixel value hv-niter resulting from the filter processing is less than the minimum value 
(c, h, m) at a step S29. When the image refresher 12a determines that the pixel value 
hv-fiiter resulting from the filter processing is less than the minimum value (c, h, m), the 
processing proceeds to a step S30. 

The image refresher 12a replaces the pixel value h V -mter with the minimum 
value (c, h, m) at the step S30. At the step S28, the image refresher 12a stores in the 
buffer 13a the minimum value (c, h, m) replacing the pixel value hv-mter as the pixel 
value of the pixel h. 

When the image refresher 12a determines at the step S29 that the pixel value 
hv-niter resulting from the filter processing is not less than the minimum value (c, h, m), 
the processing proceeds to the step S28. The image refresher 12a stores the pixel 
value hv-niter resulting from the filter processing in the buffer 13a as the pixel value of 
the pixel h. The processing then returns to the step S2 1 . 

Thus, when the processing of the step S23 shows that the value of the 
vertical-direction central pixel energy is the predetermined threshold value or more 
(when the unprocessed pixel has no strong correlation with the pixels in the vertical 
direction), the maximum value (c, h, m) and the minimum value (c, h, m) obtained by 
the processing of the step S24 are considered to be a maximum value and a minimum 
value in a local area of the pixels c, h, and m, as shown in Fig. 5. When the pixel 
value resulting from the filter processing at the step S25 falls within a range defined 
by the maximum value and the minimum value, the pixel value resulting from the 
filter processing is stored in the buffer 13a. The pixel value below that range is 
replaced with the minimum value, and the minimum value is stored in the buffer 13a. 
The pixel value above that range is replaced with the maximum value, and the 
maximum value is stored in the buffer 13a. When the processing of the step S23 
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shows that the value of the vertical-direction central pixel energy is not the 
predetermined threshold value or more, that is, when the unprocessed pixel has a 
strong correlation with the pixels in the vertical direction, the original pixel value is 
stored in the buffer 1 3a as it is. 

The description will return to the flowchart of Fig. 2. 

After performing the one-dimensional vertical edge enhancement processing 
at the step S2, the image refresher 12a performs one-dimensional horizontal edge 
enhancement processing at a step S3. 

The one-dimensional horizontal edge enhancement processing will be 
described in the following with reference to a flowchart of Fig. 6. 

At a step S41, the image refresher 12a determines whether the pixels of the 
field data stored in the buffer 13a that have been subjected to the one-dimensional 
vertical edge enhancement processing include a pixel that has not been subjected to 
the one-dimensional horizontal edge enhancement processing. When the image 
refresher 12a determines that there is an unprocessed pixel, the processing proceeds to 
a step S42. 

At the step S42, the image refresher 12a retrieves the unprocessed pixel and 
then calculates the horizontal-direction central pixel energy of the retrieved 
unprocessed pixel. For example, supposing that there are field data as shown in Fig. 
7, in which pixels a to e, pixels f to j, and pixels k to o are disposed on horizontal- 
direction lines y + 1, y, and y - 1, respectively, the horizontal-direction central pixel 
energy En-h of an A area around the pixel h (area enclosed by a solid line in the figure) 
is obtained by the following equation: 

E H - h = |(d + i + n)-(b + g + l)| ...(3) 

where b, d, g, i, 1, and n are pixel values of the pixels b, d, g, i, 1, and n, 
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respectively. Thus, horizontal-direction central pixel energy Eh of the equation (3) is 
the absolute value of the difference between the sums of the pixel values on the right 
line and the left line with the unprocessed pixel at the center. Therefore, when the 
right pixels are correlated with the left pixels, there is no large difference between the 
pixel values of the right pixels and the left pixels, and accordingly the horizontal- 
direction central pixel energy is low. On the other hand, when the right pixels are not 
correlated with the left pixels, there is often a large difference between the pixel 
values of the right pixels and the left pixels, and consequently the horizontal-direction 
central pixel energy is high. 

The image refresher 12a obtains the horizontal-direction central pixel energy 
E H of the unprocessed pixel by calculating the above equation (3). 

At a step S43, the image refresher 12a determines whether the obtained 
horizontal-direction central pixel energy Eh is at a predetermined value or more. 
When the image refresher 12a determines that the obtained horizontal-direction 
central pixel energy E H is at the predetermined value or more, that is, when the image 
refresher 12a determines that the unprocessed pixel has no strong correlation with the 
pixels in the horizontal direction, the processing proceeds to a step S44. 

At the step S44, the image refresher 12a obtains a maximum value and a 
minimum value by comparison of the pixel values of the five horizontal pixels 
including the unprocessed pixel. Specifically, when the unprocessed pixel is the pixel 
h, as shown in Fig. 7, for example, the image refresher 12a reads the pixel values of 
the horizontal pixels f to j (B area enclosed by a broken line in Fig. 7), including the 
pixel h, and then obtains a maximum value (f, g, h, i, j) and a minimum value (f, g, h, i, 
j) of the pixel values, as shown in Fig. 8. 

At a step S45, the image refresher 12a performs one-dimensional horizontal 
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filter processing, as shown in Fig. 8, on the pixels f to j in the B area shown in Fig. 7. 
Specifically, the processing of the one-dimensional horizontal filter is (1/4 - a/2, 1/4, 
a, 1/4, 1/4 - a/2) (1 < a □ 2), and a pixel value h H -niter resulting from the filter 
processing is obtained by a calculation expressed by the following equation (4): 

h H -fiiter = f x (1/4- a/2) + g x i/4 + hxa+i x 1/4+j x(i/4- 
a/2) ... (4) 

where a is a constant that can be set arbitrarily in a range of 1 < a □ 2, and 
thus enables adjustment of a degree of edge enhancement. 

At a step S46, the image refresher 12a compares the pixel value h H -fiiter 
resulting from the filter processing with the maximum value (f, g, h, i, j), and thereby 
determines whether the pixel value hH-niter resulting from the filter processing is more 
than the maximum value (f, g, h, i, j). When the image refresher 12a determines that 
the pixel value h H .fiiter resulting from the filter processing is more than the maximum 
value (f, g, h, i, j), the image refresher 12a replaces the pixel value h H -f,i t er with the 
maximum value (f, g, h, i, j) at a step S47. 

At a step S48, the image refresher 12a stores in the field memory 1 le the 
maximum value (f, g, h, i, j) replacing the pixel value hn-fiiter as the pixel value of the 
pixel h. The processing then returns to the step S41 to repeat the same processes until 
it is determined that the one-dimensional horizontal edge enhancement processing is 
performed on all the pixels. 

When the image refresher 12a determines that the horizontal-direction central 
pixel energy Eh is not at the predetermined value or more, that is, when the image 
refresher 12a determines that the unprocessed pixel has a strong correlation with the 
pixels in the horizontal direction at the step S43, the processing proceeds to the step 
S48. The image refresher 12a then stores the pixel value of the pixel h in the field 
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memory 1 le as it is. The processing returns to the step S4L 

When the image refresher 12a determines at the step S46 that the pixel value 
hH-niter resulting from the filter processing is not more than the maximum value (f, g, h, 
i, j), the image refresher 12a compares the pixel value h H .fiiter resulting from the filter 
processing with the minimum value (f, g, h, i, j), and thereby determines whether the 
pixel value h H -fiiter resulting from the filter processing is less than the minimum value 
(f, g, h, i, j) at a step S49. When the image refresher 12a determines that the pixel 
value h H -fiiter resulting from the filter processing is less than the minimum value (f, g, h, 
i, j), the processing proceeds to a step S50. 

The image refresher 12a replaces the pixel value h H -fiiter with the minimum 
value (f, g, h, i, j) at the step S50. At the step S48, the image refresher 12a stores in 
the field memory 1 le the minimum value (f, g, h, i, j) replacing the pixel value h H -fiiter 
as the pixel value of the pixel h. 

When the image refresher 12a determines at the step S49 that the pixel value 
hH-fiiter resulting from the filter processing is not less than the minimum value (f, g, h, i, 
j), the processing proceeds to the step S48. The image refresher 12a stores the pixel 
value h H -fiiter resulting from the filter processing in the field memory 1 le as the pixel 
value of the pixel h. The processing then returns to the step S41 . 

Thus, when the processing of the step S43 shows that the value of the 
horizontal-direction central pixel energy is the predetermined threshold value or more 
(when the unprocessed pixel has no strong correlation with the pixels in the horizontal 
direction), the maximum value (f, g, h, i, j) and the minimum value (f, g, h, i, j) 
obtained by the processing of the step S44 are considered to be a maximum value and 
a minimum value in a local area of the pixels f, g, h, i, and j, as shown in Fig. 8. 
When the pixel value resulting from the filter processing at the step S45 falls within a 
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range defined by the maximum value and the minimum value, the pixel value 
resulting from the filter processing is stored in the field memory lie. The pixel value 
below that range is replaced with the minimum value, and the minimum value is 
stored in the field memory lie. The pixel value above that range is replaced with the 
maximum value, and the maximum value is stored in the field memory lie. When 
the processing of the step S43 shows that the value of the horizontal-direction central 
pixel energy is not the predetermined threshold value or more, the original pixel value 
is stored in the field memory 1 le as it is. 

The description will return to the flowchart of Fig. 2. 

The image refresher 12b reads field data with a delay corresponding to one 
field from the field memory lib in the same timing as the image refresher 12a. 
Similarly to the image refresher 12a, the image refresher 12b performs the one- 
dimensional vertical edge enhancement processing and the one-dimensional horizontal 
edge enhancement processing at the steps S2 and S3, respectively, in Fig. 2. The 
image refresher 12b thereby generates edge-enhanced field data, and then stores the 
edge-enhanced field data in the field memory 1 If. The processing of the image 
refresher 12b is the same as that of the image refresher 12a, and therefore its 
description will be omitted. 

After the one-dimensional horizontal edge enhancement processing is 
performed at the step S3, the inter-field interpolating unit 15 performs inter-field 
interpolation processing at a step S4. 

The inter-field interpolation processing will be described in the following 
with reference to the flowchart of Fig. 9. 

At a step S61, the inter-field interpolating unit 15 reads field data stored in 
the field memories lie and llf. At a step S62, the inter-field interpolating unit 15 
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combines the field data read from the field memory lie with the field data read from 
the field memory 1 If and then stores the result in the interpolation memory 16b. 

Specifically, when field data read at a time t is stored in the field memory lie, 
for example, the field data stored in the field memory 1 1 f is read at a time t - 1 . Thus, 
the position of pixels arranged in a vertical direction in the field data read at the time t 
is displaced from the position of pixels arranged in a vertical direction in the field data 
read at the time t - 1, as is shown by pixels A and B in the field data of the time t and 
pixels C and D in the field data of the time t - 1 in Fig. 10. In this case, the position 
of pixels to be interpolated in the field data of the time t is the same as the position of 
the pixels in the field data of the time t - 1. 

Accordingly, the inter-field interpolating unit 1 5 combines the field data of 
the time t - 1 as pixels E and F in lines between the pixels of the field data of the time 
t, as by copying the pixels C and D in Fig. 10, for example. The inter-field 
interpolating unit 1 5 thereby generates a progressive picture and then stores the 
progressive picture in the interpolation memory 16b. 

Thus, the pixel values of pixels in a still image area are not varied greatly 
with time and therefore can be generated by such inter- fie Id interpolation processing. 

The description will return to the flowchart of Fig. 2. 

After the inter-field interpolation processing is performed at the step S4, the 
intra-field interpolating unit 14 performs intra-field interpolation processing at a step 
S5. 

The intra-field interpolation processing of the intra-field interpolating unit 14 
will be described in the following with reference to Fig. 11. 

At a step S71, the intra-field interpolating unit 14 retrieves field data stored in 
the field memory 1 le to determine whether the field data has an uninterpolated pixel. 
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When the intra-field interpolating unit 14 determines that the field data has an 
uninterpolated pixel, the processing proceeds to a step S72. 

At the step S72, the intra-field interpolating unit 14 reads the position of the 
pixel to be interpolated. At a step S73, the intra-field interpolating unit 14 obtains 
proximity energy E nea r near the position of the pixel to be interpolated. When pixels a 
to e and pixels g to k are present on horizontal lines y - 1 and y + 1, respectively, and 
a pixel f is to be interpolated on a line y between the lines y - 1 and y + 1 , as shown in 
Fig. 12, for example, the proximity energy E nea r near the position of the pixel to be 
interpolated is obtained from the following equation (5): 

Enear-f = |a - k| + |b - j| + |c - i| + |d - h| + |e - 
g| ...(5) 

Here, a to k are pixel values of the pixels a to k. 

Thus, the proximity energy E nea r near the position of the pixel to be 
interpolated is the sum of the absolute values of the differences between the pixel 
values of pixels that are present on the lines over and under the position of the pixel to 
be interpolated and that are at positions symmetrical with respect to the position of the 
pixel to be interpolated. Therefore, the higher the value of the proximity energy Enear, 
the higher the possibility that the pixel to be interpolated forms an edge portion. On 
the other hand, the lower the value of the proximity energy Enear, the higher the 
possibility that the pixel to be interpolated is not present in an edge or a texture 
portion. 

At a step S74, the intra-field interpolating unit 14 determines whether the 
proximity energy E nea r is higher than a predetermined value. When the intra-field 
interpolating unit 14 determines that the proximity energy E nea r is higher than the 
predetermined value (when the intra-field interpolating unit 14 determines that there is 
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a high possibility that the pixel to be interpolated forms an edge portion), the 
processing proceeds to a step S75. 

At the step S75, the intra-field interpolating unit 14 obtains a combination of 
pixels in a direction of high correlation from the pixels near the pixel to be 
interpolated. Specifically, the correlation is the absolute value of the difference 
between the pixel values of pixels present at positions symmetrical with respect to the 
point of the pixel to be interpolated. For example, as shown in Fig. 12, when the 
pixel f is to be interpolated, the correlations to be obtained are |a - k|, |b - j|, |c - i|, |d - 
h|, and |e - g|. After obtaining the correlations, a combination having the lowest 
value is detected and set as a combination of pixels with the highest correlation. 
Since a high correlation means no significant change in the pixel value, the lower the 
absolute value of the pixel value difference, the higher the correlation. When |b - j| 
has the lowest value, for example, the direction of high correlation shown by a 
combination of pixels is the direction of a straight line connecting the pixels b and j. 

At a step S76, an interpolated pixel is generated from a combination of pixels 
in a detected direction. When the combination of the pixels b and j shown in Fig. 12 
is the combination of pixels in the direction of the highest correlation, for example, the 
pixel value of the interpolated pixel f is obtained from the following equation (6): 

f=l/2x(b+j) ...(6) 

At a step S77, the intra-field interpolating unit 14 determines whether the 
interpolated pixel value is changed properly with respect to the pixel values in a 
vertical direction. When interpolating the pixel f shown in Fig. 12, for example, the 
intra-field interpolating unit 14 calculates a determination equation J expressed as the 
following equation (7), and then makes the above determination depending on 
whether the result of the calculation is positive or negative: 
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J = (f-c)x(i-f) ...(7) 

Here, c and i are the pixel values of the pixels c and i present over and under 
the interpolated pixel f. 

Thus, there is generally a high possibility that the pixel value monotonously 
increases or monotonously decreases in the vertical direction. This is used to 
determine whether the pixel value is changed properly. When the pixel value 
monotonously increases or monotonously decreases, the determination equation J 
gives a positive value, whereas when the pixel value does not increase monotonously 
or does not decrease monotonously, the determination equation J gives a negative 
value. 

Accordingly, when the determination equation J gives a positive value, the 
intra-field interpolating unit 14 determines that the pixel value is changed properly in 
the vertical direction, and then the processing proceeds to a step S78. 

At the step S78, the intra-field interpolating unit 14 stores in the interpolation 
memory 16a the pixel value generated from the combination of pixels in the direction 
of high correlation as the pixel value of the interpolated pixel. The processing returns 
to the step S71 to repeat the above processing steps until all pixels are interpolated. 

When the intra-field interpolating unit 14 determines at the step S74 that the 
proximity energy E nea r is not higher than the predetermined threshold value, the intra- 
field interpolating unit 14 generates a pixel value by linear interpolation using an 
upper and a lower pixel and stores the generated pixel value in the interpolation 
memory 16a at a step S79. Specifically, when interpolating the pixel f shown in Fig. 
12, for example, the intra-field interpolating unit 14 obtains a pixel value from the 
pixel values of the pixels c and i present over and under the pixel f by the linear 
interpolation represented by the following equation (8), and then stores the obtained 
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pixel value in the interpolation memory 16a as the pixel value of the interpolated pixel. 

The processing returns to the step S71 to repeat the above processing steps until all 

pixels to be interpolated are interpolated. 

f= l/2(c + i) ...(8) 

When the intra-field interpolating unit 14 determines at the step S77 that the 

pixel value is not changed properly in the vertical direction, that is, when the 

determination equation J gives a negative value, the processing proceeds to the step 

S79. 

The description will return to the flowchart of Fig. 2. 

After the intra-field interpolation processing is performed at the step S5, the 
still/moving image area determining unit 17 performs still/moving image area 
determination processing at a step S6. 

The still/moving image area determination processing will be described with 
reference to the flowchart of Fig. 13. 

At a step S91, the still/moving image area determining unit 17 determines 
whether there is a pixel that has not been subjected to the still/moving image area 
determination processing. When the still/moving image area determining unit 17 
determines that there is a pixel that has not been subjected to the still/moving image 
area determination processing, the processing proceeds to a step S92. 

At the step S92, the still/moving image area determining unit 17 retrieves the 
pixel that has not been subjected to the still/moving image area determination 
processing as a remarked pixel and then reads the position of the remarked pixel. At 
a step S93, the still/moving image area determining unit 17 calculates absolute values 
of temporal differences between inter-field and intra-field pixels (temporal differences 
between pixel values) near the remarked pixel. Specifically, as shown in Fig. 14, for 
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example, suppose that pixels a and b are present in field data of a time t as pixels 
arranged in a vertical direction, and a remarked pixel s (pixel to be interpolated) is 
present between the pixels a and b. Also, suppose that pixels c and d are present in 
field data of a time t - 1, pixels e and f are present in field data of a time t - 2, and 
pixels g and h are present in field data of a time t - 3. In this case, the pixels of the 
field data of the time t and the pixels of the field data of the time t - 2 are present on 
the same vertical line, and the pixels of the field data of the time t - 1 and the pixels of 
the field data of the time t - 3 are present on the same vertical line. The alternating 
lines are displaced by one line in the vertical direction in positional relation to each 
other. The field data of the times t to t - 3 in Fig. 14 indicates pixel positions on the 
same lines in a horizontal direction. 

In this state, absolute values of inter-field temporal differences are |a - c| and 
|b - c|, and absolute values of intra-field temporal differences are |a - e|, |c - g|, and |b 
- f|. The still/moving image area determining unit 17 reads the field data of the times 
t to t - 3 from the field memories 1 la to 1 Id, and from the field data, the still/moving 
image area determining unit 17 calculates the absolute values of inter-field temporal 
differences (|a - c| and |b - c|) and the absolute values of intra-field temporal 
differences (|a - e|, |c - g|, and |b - f|). 

At a step S94, the still/moving image area determining unit 17 obtains a 
maximum value of the absolute values of the inter-field temporal differences and a 
maximum value of the absolute values of the intra-field temporal differences. 

At a step S95, the still/moving image area determining unit 17 determines 
whether the maximum value of the absolute values of the intra-field temporal 
differences is higher than 1/3 of the maximum value of the absolute values of the 
inter-field temporal differences. When the still/moving image area determining unit 
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1 7 determines that the maximum value of the absolute values of the intra-field 
temporal differences is higher than 1/3 of the maximum value of the absolute values 
of the inter-field temporal differences, the still/moving image area determining unit 1 7 
stores the remarked pixel as a moving image pixel at a step S96. The processing 
returns to the step S91 to repeat the same processing steps until all pixels are subjected 
to the still/moving image area determination processing. 

When the still/moving image area determining unit 17 determines at the step 
S95 that the maximum value of the absolute values of the intra-field temporal 
differences is not higher than 1/3 of the maximum value of the absolute values of the 
inter-field temporal differences, the still/moving image area determining unit 17 
determines at a step S97 whether a result of multiplication of two inter-field temporal 
differences is negative and an absolute value of a result of addition of the two inter- 
field temporal differences is lower than a predetermined threshold value. When the 
still/moving image area determining unit 17 determines that the result of 
multiplication of the two inter-field temporal differences is negative and the absolute 
value of the result of addition of the two inter-field intra-field temporal differences is 
lower than the predetermined threshold value, the processing proceeds to a step S98. 

At the step S98, the still/moving image area determining unit 17 determines 
whether one of the absolute values of the two inter-field temporal differences is higher 
than a predetermined minimum threshold value and higher than 1/2 of the other 
absolute value. When the still/moving image area determining unit 17 determines 
that one of the absolute values of the two inter-field temporal differences is higher 
than the predetermined minimum threshold value and higher than 1/2 of the other 
absolute value, the processing proceeds to the step S96. When the still/moving image 
area determining unit 17 determines that one of the absolute values of the two inter- 
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field temporal differences is not higher than the predetermined minimum threshold 
value and higher than 1/2 of the other absolute value, the processing proceeds to a step 
S99. 

At the step S99, the still/moving image area determining unit 17 determines 
whether one of the absolute values of the two inter-field temporal differences is higher 
than a predetermined maximum threshold value and higher than 1/4 of the other 
absolute value. When the still/moving image area determining unit 17 determines 
that one of the absolute values of the two inter- field temporal differences is higher 
than the predetermined maximum threshold value and higher than 1/4 of the other 
absolute value, the processing proceeds to the step S96. When the still/moving image 
area determining unit 17 determines that one of the absolute values of the two inter- 
field temporal differences is not higher than the predetermined maximum threshold 
value and higher than 1/4 of the other absolute value, the processing proceeds to a step 
S100. 

At the step SI 00, the still/moving image area determining unit 17 stores the 
remarked pixel as a pixel in a still image area. The processing proceeds to the step 
S91. 

When the still/moving image area determining unit 1 7 determines at the step 
S97 that the result of multiplication of the two inter-field temporal differences is not 
negative or the absolute value of the result of addition of the two inter-field temporal 
differences is not lower than the predetermined threshold value, the processing 
proceeds to the step SI 00. 

Thus, when the still/moving image area determining unit 1 7 determines by 
the processing of the step S95 that the maximum value of the absolute values of the 
intra-field temporal differences is higher than 1/3 of the maximum value of the 
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absolute values of the inter-field temporal differences, the remarked pixel is 
considered to be a moving image pixel. Otherwise, under a condition that the result 
of multiplication of the two inter-field temporal differences is negative and the 
absolute value of the result of addition of the two inter-field temporal differences is 
lower than the predetermined threshold value, the remarked pixel also is considered 
to be a moving image pixel when one of the absolute values of the two inter-field 
temporal differences is higher than the predetermined minimum threshold value and 
higher than 1/2 of the other absolute value or when one of the absolute values of the 
two inter-field temporal differences is higher than the predetermined maximum 
threshold value and higher than 1/4 of the other absolute value. Otherwise, the 
remarked pixel is considered to be a still image pixel. 

The description will return to the flowchart of Fig. 2. 

After the still/moving image area determination processing is performed at 
the step S6, the selector 18 performs selection processing at a step S7. 

The selection processing will be described with reference to a flowchart of 

Fig. 15. 

At a step Sill, the selector 18 determines whether there is an uninterpolated 
remarked pixel. At a step SI 12, the selector 1 8 reads the position of the remarked 
pixel. At a step SI 13, the selector 18 inquires of the still/moving image area 
determining unit 17 whether the remarked pixel corresponding to the read pixel 
position is a moving image pixel. When the selector 1 8 thereby determines that the 
remarked pixel is a moving image pixel, the processing proceeds to a step SI 14. 

At the step SI 14, the selector 18 reads a pixel value which corresponds to the 
read pixel position and is interpolated as a pixel in a moving image area and stored in 
the interpolation memory 16a and then inserts and stores the pixel value as a pixel to 
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be outputted as a progressive picture. The processing returns to the step Sill. 

When the selector 18 determines at the step SI 13 that the remarked pixel is 
not a moving image pixel, that is, the remarked pixel is a still image pixel, the selector 
1 8 reads a pixel value which corresponds to the read pixel position and is interpolated 
as a pixel in a still image area and stored in the interpolation memory 16b and then 
inserts and stores the pixel value as a pixel to be outputted as a progressive picture at a 
step S 1 1 5. The processing returns to the step Sill. The processing from the step 
Sill on down is repeated until all pixels are interpolated. 

Thus, on the basis of the read position of pixels of interest, the selector 18 
inquires of the still/moving image area determining unit 1 7 whether each of the pixels 
of interest is stored as a pixel in a still image area or stored as a pixel in a moving 
image area. When the remarked pixel is a pixel in a moving image area, the selector 
1 8 inserts a pixel value interpolated as a moving image pixel by the intra-field 
interpolating unit 14 and stored in the interpolation memory 16a. When the remarked 
pixel is a pixel in a still image area, the selector 18 inserts a pixel value interpolated as 
a still image pixel by the inter-field interpolating unit 1 5 and stored in the 
interpolation memory 16b. 

The description will return to the flowchart of Fig. 2. 

After the selection processing at the step S7 is completed, the selector 18 
outputs a generated progressive picture to an apparatus in a succeeding stage not 
shown in the figure at a step S8. 

While the above description has been made by taking a case where the image 
refresher 12a uses a three-tap vertical filter and a five-tap horizontal filter, the image 
refresher 12a also may use a three-tap vertical filter and a three-tap horizontal filter. 
In this case, as shown in Fig. 16, the vertical filter may be configured for processing 
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as (1/4 - a/2, a + 1/2, 1/4 - a/2), and the horizontal filter may be configured for 
processing as (1/4 - a/2, a + 1/2, 1/4 - a/2), for example. The horizontal filter 
performs filter processing on every other pixel value of five read pixel values (for 
example on pixels f, h, and j when pixels f, g, h, i, and j are sequentially read). Also, 
a combination of a three-tap vertical filter and a five -tap horizontal filter and a 
combination of a three-tap vertical filter and a three-tap horizontal filter both may be 
made available and selectable by the user. In addition, a vertical filter and a 
horizontal filter having a different number of taps may be used. Furthermore, pieces 
of field data or pixels used for interpolation do not need to be continuous; for example, 
one in every two or more pieces of field data or pixels may be used. 

Furthermore, in addition to the equation (5), the following equation (9) may 
be used as an arithmetic equation for obtaining the proximity energy near an 
unprocessed pixel, and selection between the equations (5) and (9) may be made by 
the user. 

Enear-f = |a - g| + |b - h| + |c - i| + |d - j| + |e - 
k| ...(9) 

According to the present invention described above, in processing for 
converting an interlaced image into a progressive image, it is possible to enhance 
properly an edge even in an image with an insufficient high-frequency component, 
improve vertical resolution in a still image area, and smoothly interpolate an oblique 
edge in a moving image area. It is also possible to prevent an error in conversion 
processing. 

The series of processing steps described above may be carried out not only 
by hardware but also by software. When the series of processing steps is to be 
carried out by software, a program forming the software is installed from a recording 
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medium onto a computer that is incorporated in special hardware or a general-purpose 
personal computer that can perform various functions by installing various programs 
thereon, for example. 

Fig. 1 8 shows a configuration of an embodiment of a personal computer when 
the image processing apparatus is realized by software. A CPU 101 of the personal 
computer controls the entire operation of the personal computer. When a command is 
inputted from an input unit 106 formed by a keyboard, a mouse and the like by the 
user via a bus 104 and an input/output interface 105, the CPU 101 executes a program 
stored in a ROM (Read Only Memory) 102 in response to the command. Also, the 
CPU 101 loads into a RAM (Random Access Memory) 103 a program read from a 
magnetic disk 1 1 1 , an optical disk 1 12, a magneto-optical disk 1 13, or a 
semiconductor memory 114 connected to a drive 110 and installed in a memory unit 
108 and then executes the program. The above-described functions of the image 
processing apparatus are thus realized by software. The CPU 101 also controls a 
communication unit 109 for communication with the outside and thereby sends and 
receives data. 

Examples of the recording medium on which the program is recorded include 
not only program-recorded package media distributed to users to provide the program 
separately from computers, such as magnetic disks 1 1 1 (including floppy disks), 
optical disks 112 (including CD-ROM (Compact Disk-Read Only Memory) and DVD 
(Digital Versatile Disk)), magneto-optical disks 113 (including MD (Mini-Disk)), or 
semiconductor memories 1 14, as shown in Fig. 18, but also a hard disk included in the 
memory unit 108 and the ROM 102 storing the program and supplied to the user in a 
state of being preincorporated in the computer. 

It is to be noted that in the present specification, the steps describing the 
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program recorded on a recording medium include not only processing steps carried 
out in time series in the described order but also processing steps carried out in 
parallel or individually and not necessarily in time series. 

The apparatus, the method, and the program for image processing according 
to the present invention interpolate a pixel of an image of a progressive system using a 
pixel of a field of an image of an interlacing system to be converted and a pixel of a 
different field, interpolate a pixel of the image of the progressive system using pixels 
within the field of the image of the interlacing system to be converted, determine 
whether a pixel to be interpolated of the image of the progressive system is a moving 
image pixel or a still image pixel, and select either the intra-field interpolated pixel or 
the inter-field interpolated pixel on the basis of a result of the determination. 
Therefore, in processing for converting an interlaced image into a progressive image, 
it is possible to enhance an edge even in an image with an insufficient high-frequency 
component, improve vertical resolution in a still image area, and smoothly interpolate 
an oblique edge in a moving image area. It is also possible to prevent an error in 
conversion processing. 

The entire disclosure of Japanese Patent Application No. 2001-006365 filed 
on Jan. 15, 2001 including the specification, claims, drawings and summary is 
incorporated herein by reference in its entirety. 
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